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iBSTBACT 

The Eesearch and Development Utilization Program 
(EDO) was intended to apply research and development products to 
local school 1 problems; to develop a problem-solving and product 
selection process usable by local schools; and to organize a linkage 
system making national, state, and other external resources available 
to local schcol personnel. The service delivery system of the EDU 
program operated through seven projects in 20 states and served over 
300 schools or school districts over the period 1976-79. Qualitative 
research methods, that is, case study reports, site visit material,, 
and coded written reports, are the data sources for this report. The 
discussion of the preliminary findings places equal eaphasis on 
products and problem-solving processes. Product characteristics 
positively related to successful programs are validated 
effectiveness, complexity, and direct impact on pupils. 
Problem-solving process characteristics positively related to 
successful programs are linking agent behavior as measured by the 
amount of time spent by the agent, amount of influence of the agent, 
and the agent's adoption of a facilitator or process role. 
<Author/MLF) 
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INTRODUCTION 

In June of 1976, the National Institute of Education (NIE) established the 
Research and Development Utilization (RDU) program as a new action-research 
effort in dissemination. This program was designed to: 

© apply R&D products, or ideas , to school problems; 

• develop a problem-solving process / whereby schools 
would systematically identify such problems and 
select and implement new ideas; and 

• organize a linkage system / whereby national/ state, 
and other external resources would be made available 
to school personnel. 

The RDU program is unusual among federally funded dissemination strate- 
gies because of its dual commitment to the dissemination and use of R&D 
products and to the development of local school capabilities to solve prob- 
lems through the use of externally generated knowledge. Other federal 
programs, including some of those which you will hear about in this panel 
discussion/ have tended to concentrate on either product dissemination or 
local capacity building/ but not on an integrated model for combining the 
two. The core of the RDU strategy was to provide each participating site, 
which was either a school or a district, with assistance in the following 
sequence of activities: 

• identification of a problem or set of problems; 

• examination of alt^ native solutions to the problem, 
focusing particularly on the products of educational 
R&D; 

• selection of a specific solution to address the prob- 
lem; 

9 implementation of the solution; 

• evaluation and incorporation of both the solution and 
the problem solving process. 



The service delivery system of the RDU program operated through seven 
projects , each of which coordinated a network of organizations and individ- 
uals involved in the provision of services and information to local schools 
and school districts. As a whole, the seven projects operated in 20 states 
and served over 300 schools or school districts over a three-year period 
(1976-1979). Each of the projects selected and made available a pool of R&D 
products , which was also referred to as a knowledge base. These knowledge 
bases were developed as a resource for identifying solutions to match client 
school needs. The projects also deployed " linking agents " who coordinated 
the services provided to local schools and school districts, and who helped 
guide the local school personnel in a school improvement process. Each 
project supported two or more linkers. Most operated out of an intermediate 
service agency, or a state education agency, a i each served a specific set 
of local schools or school districts. The range of a linker's possible roles 
included facilitating the decision-making process by clarifying goals and 
providing leadership, and mediating among autonomous organizations ^hose re- 
sources and services required coordination. The seven projects were region- 
ally distributed, and involved the following: 

• The Northwest Reading Consortium , a consortium of four 
states in the Northwest — Washington, Oregon, Alaska, and 
Idaho. 

• The Georgia state Department of Education . 

• The Pennsylvania Department of E d ucation . 

• The National Education Associ a tion , which operated its 
project in collaboration with the State Education Agen- 
cies and corresponding state education associations in 
12 states. 

• The Florida Department of Education . 

• Th : Michigan Department of Education . 

• The NETWORK/Consortium , a non-profit research and ser- 
vice organization which coordinated agencies in six 
states. 



In November ' 1977, Abt Associates Inc., a social science research firm 
based in Cambridge, Massachusetts, was contracted to conduct a study of the 
RDU program. The study addresses six major issues: 

• How relationships are managed between various agencies 
which have the expertise and resources to help local 
schools solve problems; 

m To what degree an intervention program such as RDU can 
help schools overcome barriers to successful problem 
solving (such as limited access to information, or lack 
of planning skils r etc.); 

• To what degree the products of educational r&d are rel- 
vant to the problems and contexts of local schools; 

• What the impact is of the products of educational R&D 
once they have been adopted and implemented; 

• What factors contribute to the institutionalization of 
the RDU approach within a variety of organizations; 

• How linking agents coordinate the flow of external re- 
sources to schools, and whether this helps the schools 
solve problems. 

All of the seven projects have completed the federally supported service 
delivery part of their activities. However, our research efforts are just 
beginning to bear fruit. While we have already produced a number of interim 
reports (Chabotar and Kell, 1978; Louis et al. , 1979; Yin, Gwaltney and 
Louis, 1980; Spencer and Louis, 1979; Kell and Louis, 1979) we have only 
recently ^ completed the data collection that will allow us to assess the 
impact of the RDU program on schools. Thus, the following discussion of 
findings and policy implications is necessarily preliminary and limited.* 

PRELIMINARY FINDINGS ABOUT THE R&D UTILIZATION PROGRAM 

Because a key feature of the program design was its equal emphasis upon 
products and problem-solving processes, the discussion of our preliminary 
findings will be divided accordingly. 

*The final reports from this study will be available in the winter and 
spring of 1981. 



Findings Related to the Appli c ability arid Impact of R&D P r oducts on 
Schools " "" 

The intent of the R&D Utilization Program, as it was expressed in the 
RFP and other early documents, was to help schools solve educational problems 
through the use of existing research and development based "products" (cur- 
riculum innovations). The original RFP indicated that respondents should 
limit their knowledge bases to products related to either basic skills or 
career e ducation . In addition, the RFP indicated that contractors should 
emphasize quality control over the products. The products should show 
evidence of effectiveness and evidence of transportability from one site to 
another. Thus, throughout the early portions of the RDU program, a great 
deal of attention within each project was given to developing a knowledge 
base that contained acceptable products and designing a process for deliver- 
ing these products to the schools. 

Because the RFP specified empirical validation as the criterion of qual- 
ity, a major issue for e^h of the projects had to do with the product's ac- 
ceptability by evaluative standards.* Our analysis, however, suggests that 
many schools eventually were presented with numerous non-validated products 
for adoption, including the following examples: 

• Two projects provided assistance in career education, a cur- 
riculum topic that had been identified in the RFP but in which 
the bulk of existing products had not been validated. As one 
result, only 20 percent of one project's products- were believed 
to be validated. 

• Several projects provided assistance on topics that were not 
identified in the RFP and in which available products were 
also mainly non- validated. 

• In other projects, individual schools were reported to have 
adopted products that had not been validated. 

♦Projects were initially allowed to disseminate non-validated products 
only if they developed and implemented validation procedures themselves. 
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In only two projects did it appear that schools had adopted only vali- 
dated products. ' The fact that the svven product pools collectively offered 
nearly 600 products for sites to consider (of which over 100 were adopted by 
one or more of the sites served during the project) made it impossible for us 
to determine a precise "quality" measure for each of the knowledge bases. 
However, an inferential use of secondary data and reports from project staff 
members leads us to estimate that possibly 60 percent of the adoptions made 
by RDU sites involved non-validated products. This slippage between the in- 
tent of the original RDU mandate/ as represented in the RFP and other early 
documents, and the actual operation of the projects occurred despite' enor- 
mous efforts by each of the seven projects to develop a knowledge base 
responsive to the program 1 s design. 

The primary reasons for this slippage wvsre: 

• the adequacy of the pool of educational products; and 

• site (school or district) preferences* 

As noted above, in m^ny areas there are simply not a sufficient number of 
formally validated educational products to meet the needs of local schools. 
Among the topics treated by the R&D Utilization projects in which few accept- 
able products were found were the following: career education, district or 
school-wide planning, and in-se* /ice training tor teachers. In addition, the 
area of secondary school curricula, particularly in the area of basic skills, 
produced very few tested educational products that met school needs. In order 
to be responsive to the service delivery mandates of their programs, the RDU 
projects were forced in many instances to use expert judgment on surface va- 
lidity, rather than external evaluation data as the quality control basis in 
admitting products to their pools. 

6 
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In approximately 20 percent of the case*;, sites chose products that 



were not part of the project's knowledge base. in other instances, sites 
"officially" chose an "approved" product, but in fact spent most of their 
effort in developing their ovm innovation or implementing a non-validated 
product, in most cases, these "low quality" selections were made after six 
months to a year of intensive involvement between the site and the project. 
At this point it was very difficult to terminate a site solely because it 
had not found an acceptable product from within the knowledge base. In sum, 
client demands and concerns overrode the specified quality control proce- 
dures in the product delivery process. 

Despite .±ess constraints, projects were able to provide acceptable, 
attractive products to almost all of the schools involved in the program. 
Furthermore, tSe data indicate that the selected products were received 
with enthusiasm in most schools: 

• A survey of teachers indicates that approximately 80 percent 
of those who are eligible to use the product are, in fact, 
using it. 

• In addition, approximately 50 percent of the surveyed teachers 
responded enthusiastically to questions about the quality of 
the product, that is, items such as whether or not it provides 
new ideas, whether it is directly relevant to the most pressing 
problems or needs in the school. (Teachers responding that the 
item is true to a great or very great extent) . 

m Finally, principals report that a fair degree of product 
institutionalization has already occurred. Sixty-four per- 
cent, for example, indicate that the program or materials 
have been formally incorporated into school curriculum 
plans. Approximately 60 percent indicate that locally writ- 
ten guidelines for the use of materials and methods have al- 
ready been developed or will definitely be developed in the 
near future. Sixty-two percent predict extensive use of 
the materials or methods by teacners in the future. 

What characteristics of the product, if any, are related to the desired 

positive outcome cr effects of the R&D Utilization program in schools? Based 

on Abt Associates 1 analysis and coding of data from approximately 75 schools 

7 
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which were visited either by Abt Associates staff or case study writers 



employed by the seven projects , have developed five preliminary out- 
come measures which are related to the overall objectives of the program. 
Three of these reflect the overall impact of the program on the school. 
These include: 

• the scope of implementation , or the percentage of students 
and the percentage of the^student ^.ay affected by the im- 
plementation of the product? 

• institutionalization , (sustained use) of the process and 
product? and 

• a scale of organizational change , conposed of items tapping 
the amount of improvement in curriculum, materials, methods 
structure, teacher morale, pupil performance, and status of 
the problem that occurred as a result of the innovation. 

An additional two outcome variables are related to the quality of the prob- 

. ** 

iem solving process engaged in by the school. These are: 

• the level of effort devoted to problem solving; and 

• our judgments about the quality of the problem solving 
behaviors , including the degree to which they met cri- 
teria of logical and rational behavior and the degree to 
which they reflected sound group problem solving strate- 
gies. . (Intercorrelations between these five outcome mea- 
sures may be seen in Table 1.) 

If we look at the -ast line of Table 2, it may be seen that product char- 
acteristics are particularly strong predictors of the scope of implementation , 
of institutionalization, and of organizational change . The product variables 
that are the most important predictors of these outcomes of the R&D Utiliza- 
tion Program are: 

• whether or not there is any empirical evidence of the ef- 
fectiveness of the product, in other words, whether the 
product is validated; 

• whether the product is complex, in other words, does it 
have many parts, and is it designed to change many aspects 
of the school's program at once, and finally; 
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• whether the product is designed to affect pupils directly, 
in other words, whether it is related to the curriculum, 
rather than to administrative procedures in the schools, 
or to inservice. 

Each of these variables — validated effectiveness, coirplexity, and direct 
iirpact on pupils — is positively related to the outcome measures. 

One n non-f inding" is also of significance. Our data do not provide sup- 
port for the "Change Agent Study" (Berman and McLaughlin, 1979) einphasis on 
the importance of local materials development. Variables which tapped local 
materials development and the degree to which products underwent substantial 
local modifications do not effectively predict any of the RDU outcomes. We 
suspect that the local materials development variables in the "Change Agent 
Study" are best thought of as proxies for a participatory, broadly based 
process of change, (one outcome of which will be local materials development 
in a program which does not emphasize adopting externally produced curriculum 
products ) 

Policy Implications of Preliminary Produce Findings 

Our preliminary product findings have a number of significant policy im- 
plications. First, it is likely that any effort to rapidly expand the dissem- 
ination of existing R&D products will not meet many of the most pressing 
locally defined problems in schools because few products exist in these areas. 
If the federal govermoent wishes to pursue active dissemination of validated 
products to serve local school needs, they must also engage in a more active 
and rationalized product development strategy than is currently being pursued.* 

Second, no matter how well developed the product base, there will be some 
schools that choose to adopt non-validated commercial or practitioner-developed 



♦Current federal priorities do not favor curriculum development. Curric- 
ulum or product packaging and marketing support is provided almost exclusively 
to Developer/Denonstrators through the NDN program. 
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products over R&D products — or indeed to develop their own solutions. How- 
ever, based on the experience of the RDU sites,, the number of schools unre- 
ceptive to validated products is apparently quite small. In other words, 
where validated and/or R&D based products exi st and are disseminated, they 
are eagerly accepted by local school districts. Lack of prior widespread 
use does not mean a lack of relevance or potential utility of these products, 
but: rather reflects a lack of exposure. It should be emphasized that the 
selection of sites to be included in thfs R&D Utilization program did not ex- 
hibit any bias toward Fites with characteristics that might be thought to be 
predictive of strong interest in R&D products. Most were rur-al and had only 
limited experience with R&D products prior to their involvement in this pro- 
gram. Only 12% of the schools were classified by us as frequent users of new 
ideas or programs. in addition, the schools were frequently picked for in- 
clusion in the program because they exhibited problems and not because they 
exhibited a high state of readiness to change. 

Third, there is evidence to suggest that the characv. eristics of products 
are important in determining whether th«re will be a lasting and substantial 
impact upon the educational environment of the school. In this case, valida- 
tion or the existence of empirical evidence of effectiveness is apparently an 
important criterion, in our data, the validation variable also subsumes other 
characteristics of the product: Many of the validated products were supported 
by NDN developer/demonstrator grants and were better packaged, had more easily 
available materials, and often had more experienced trainers, who were avail- 
able to provide assistance both before and after implementation. This indi- 
cates that, despite the dilemmas associated with applying quality control 
procedures to the development of a knowledge hi ze, these quality control pro- 
cedures may be one of the single most important factors determining program 
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outcomes. It is also important to note that complex innovations requiring 
greater change on the part of implementing teachers are also an important 
component in producing implementation, institutionalization, and organiza- 
tional change. 

To summarize, validated products can have a big impact in the schocO im- 
provement process, and the current belief that local materials development or 
adaptation io necessary in order to produce significant and lasting change in 
schools can be challenged. A federal R&D based school improvement strategy 
should emphasize curriculum development in areas that are responsive to school 
needs not currently met by existing products, should continue to concern itself 
with quality control screening, and should engage in active marketing to en- 
sure that schools that would not normally come into contact with information 
about products will do so. 

Preliminary Findings about the Problem Solving Process 
The problem solving process in the R&D Utilization program was a rela- 
tively complex one which involved mustering both internal and external re- 
sources. Each of these will be discussed in turn. 

Internal Strategies : In most cases, the "RDU approach" involved the 
development of an internal problem solving team , which received a mandate 
from the school and district to take responsibility for defining a problem, 
examining solutions, and planning for implementation, as well as monitoring 
and evaluating implementation in some cases. These teams were/ typically, 
intended to include representatives of different role groups within the 
school. At minimum, they were to include representatives of the group that 
was intended to use the product, and representatiaves of either the central 
office or school administration. In many cases, however, the representation 
was much broader, including for example, grade level representation of all 
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teachers, principal representation, and representation of central office 
administrators and/or specialists. In some cases parents and students were 
also included. 

The problem solving process that wac used in the majority of schools in- 
volved in the R&D Utilization program was quite different from the typical 
innovation practices of most schools. Innovations in schools, according to 
our respondents, are typically introduced because a principal or an individual 
in the district office becomes enchanted with a new practice, and it is im- 
posed upon a school staff that is not equally enthusiastic. in the R&D Util- 
ization program, in contrast, influence was centered at the teacher level. 
Thus, for example, if we look at the distribution of influence in selecting 
a new solution to the identified problem, we find that central office staff 
had a high level of influence in 24 percent of the sites, as did principals 
in 40 percent of the sites, as contrasted to 65 percent of the sites in which 
teachers had a high level of influence. In addition, a survey of principals 
indicates that fewer than 6 percent believe the RDU approach to problem solv- 
ing was not distinctly different from practices they had previously engaged in. 

The function of the teams as it was interpreted by the seven projects 
was to increase the salience of, and effort devoted to, the problem solving 
process, where quality was typically defined by adherence to a rational model 
of analyzing needs, matching solutions to those needs, and of careful planning 
for and monitoring of implementation, and to increase the real influence of 
participation of parties who would be most affected by decision outcomes.* 
Looking at Table 1, we see that the level of effor t devoted to the problem 
solving process is significantly correlated with all of the other RDU program 

♦In this preliminary analysis we are aggregating measures of rationality 
and participation. Later reports will examine these aspects of the process 
separately. 

12 

14 



Table 1 

)NS (R) 01 
OF THE RDU PROGRAM 



INTERCORRELATIONS OF FIVE OUTCOMES 



Level 
of 

Scope Effort 

Scope of .27* 
Irnplemen tation (60) 

Level of Effort- 
Process 

Quality of Process 



Institutionalization 



Organizational Change 



Quality 



of Institutional- Organizational 

Proces s ization Change 



.05 .55* .39* 

(55) (54) (51) 

•44* .36* .38* 

(64) (60) (62) 



.24 .36* 
(55) (54) 

.66* 
(51) 



*Significant at the .05 level. 
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outcomes. Level of effort is particularly highly correlated with institution- 
alization (R = .36), and organizational change ( R * .38). The quality of the 
problem solving process , on the other hand, is not significantly correlated 
with either the scope of implementation, or with the level of institutional- 
ization, although it is significantly correlated with organizational change 
(R = .3). 

Further confirmation of the limited importance of an improved process in 
changing schools may be seen in Table 2. Row 3 of tnis table shows the re- 
sults of regressions which regressed the five outcomes of the RDU program 
on a series of variables which tapped the influence of the .".ocal team and of 
various role partners: teachers, principals, central office staff members 
and the Superintendent. Again we see that these process variables which re- 
flect the degree of participation of different actors in the decision making 
process have no significant predictive power for scope of implementation, 
level of effort, or institutionalization of the product and process. However, 
they do significantly predict the degree of organizational change.* 

To summarize, achieving the RDU objective of improving the quality of 
the problem solving process in a school apparently has relatively little 
effect on the short-term objectives of implementing and institutionalizing 
a new product. Improving the process may, however, have more significant, 
long-range" impacts upon the functioning of the organization and the quality 
of its curriculum. Our qualitative data suggest that changes in participation 
in problem solving, and particularly the involvement of teachers in the prob- 
lem solving process, frequently had significant impacts on teacher morale, 
communication patterns within the school, and individual staff development. 

*Not surprisingly, role partner and team influence are predictive of the 
quality of the process. This relationship is expected, since one aspect of 
quality concerned the degree to which different actor groups were effectively 
represented. 
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Table 2 



PERCENTAGE OF THE VARIANCE EXPLAINED IN FIVE OUTCOMES 
OF THE RDU PROGRAM: DATA FROM THE CONSOLIDATED , CODING FORM** 



PREDICTOR 
VARIABLES 



OUTCOME VARIABLES 



Scope 



Level 

of 
Effort 



Quality 

of 
Process 



Institution- 
alization 



Organiza- 
tional Change 



Readiness 



.19" 



.06 



.12* 



32* 



.06 



Linking Agent 
Behavior 



29* .25* 



32* 



.11" 



.19* 



Process-Role 
Partner and 
Team Influence 



,05 



.06 



.38* 



.09 



35* 



Product 

Characteristics 



.49" 



,11* .27" 



.43" 



.54' 



*F statistic for regression equation significant at .05 or better 

**A11 regressions have between 49 and 55 cases included. Equations 
were obtained through a stepwise regression procedure which optimizes 
its F ratios, (SAS Institute, 1979) Equations contain between 
1 and 4 variables. 
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In addition, product selection and adoption were often the excuse for more 
comprehensive changes in curriculum or instructional management. 

External Strategies : In addition to the development of an internal 
teacher/administrator team with a broad mandate for decision making, the 
RDU process typically involved external acto r s, or linking agents . These 
linking agents varied enormously in the amount of time they were expected to 
commit to each of the schools they worked with, ranging from 2 or 3 percent 
to 25 percent over a three-year period. In addition, they varied in terms 
of the roles they were expected to perform, jin some cases they were expected 
to serve only as a communications link between the project e>nd the site, in- 
cluding communicating the projects 1 expectations about the type of problem 
solving activities that the site would engage in. In other projects they 
were expected to take strong change agent postures in guiding the school 
through the problem solving process and in providing specialized consultant 
assistance in the topic area in which the school's problem was identified. 
While a major conponent of our study is focused on improving our under- 
standing of how and why linking agents functioned as they did in this pro- 
gram, this presentation will consider only the impact of linking agent be- 
haviors and internal strategies upon school outcomes. 

In contrast to the relatively limited inpact that the problem solving 
process (aside from level of effort) has upon product implementation, the 
linking agent and his or her activities has a broader and stronger effect. 
Row 2 of Table 2 indicates that when the five RDU outcomes e L re regressed 
on linking aagent activities and characteristics all equations are signifi- 
cant. The linking agent behaviors that seem to have the greatest impact on 
scope, level of effort, quality of the problem solving process, institution- 
alization, and organizational change have significant policy relevance: 

16 
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p The amount of time that linking agents spent with client 

schools — a variable which we have noted is highly variable — 
was extremely important as a predictor of RDU outcomes. 

• Second, the amount of influence that the linking agent had 
over client and team activities and decision making is a 
critical predictor variable. In some cases linking agents 
chose to behave neutrally at all times and to facilitate, 
but not participate. In other cases, linking agents took 
stronger normative stances, acting either as change agents, 
or becoming involved in directing or influencing the local 
teams. 

» Finally, the degree to which the linking agent adopted a 
facilitator or process role , which involved surfacing and 
resolving conflicts, providing training and' problem solving 
in group process, an'i providing technical assistance in diag- 
nosing the problem and assessing the match between innova- 
tions and solutions is a critical predictor. (Other roles 
which were less significant as predictors were acting as a 
curriculum area specialist, or simply a general support 
person. 

It is interesting to note that linking agent behaviors are least powerful 
in predicting RDU outcomes tiiat occur later in the process: institutionaliza- 
tion and organizational change * Linking agents have more influence over the 
level of effort devoted to problem solving, the quality of the problem solving 
process, and the initial scope of implementation of the product. 

The preliminary analyses suggest that, while the role of the internal 
team is important — particularly in producing longer term organizational 
changes--these beneficial outcomes of improved internal problem solving are 
unlikely to occur -in the absence of an external catalyst. Our own observa- 
tions in the 42 schools that we have visited suggest that school staffs that 
are unable to go through an effective problem solving sequence and to choose 
and implement an innovative program without the assistance of an external 
agent are typically blessed with some form of indigenous leadership which is 
to compensate for — in the initial stages, and later to develop — internal 
capacity.* 



*In most cases, we believe that the relationship between leadership pro- 
vided by the internal team and that provided by the linking agent was a key 
|v% factor in predicting success, an hypothesis that we will explore further in 

'ijj future analysis. 



Client schools and individual educators recognized the significance 
of the linking agent. When teachers , for example, were asked to rate their 
satisfaction with the assistance, services, or support provided by various 
individuals or groups, satisfaction with the linker was rated only slightly 
lower than satisfaction with the local school team. Satisfaction with 
linkers was considerably higher than satisfaction with product developers 
from whom they received training or assistance. Forty-six percent, as com- 
pared to 31 percent, were very or extremely satisfied. The rating of linkers 
as sources of assistance and support was even higher than other external pro- 
viders of assistance, such as other RDU project staff or other consultants. 
Principals indicated a similar recognition of the helpfulness of the linker 
in performing a variety of activities related to the problem solving process. 
Principals, in general, exhibited a more positive attitude toward linkers, 
which results largely from the fact that they received more direct assist- 
ance from the linkers. 

Despite local recognition that an external linking agent was an impor- 
tant part of the process, the use of the services of an external linker was 
the aspect of the process that principals were least likely to indicate they 
would use again. Thus, for example, approximately 35% of the principals in- 
dicated that they would definitely use the RDU approaches to problem identi- 
fication or identification of solutions or planning for implementation, but 
only 13% indicated that they would definitely use the services of an external 
linker in a further problem solving effort. Our site visits suggest that 
availability of linkers is the key reason for this response — the RDU linking 
roles were discontinued in' most cases after the demise 0* federal funding, 
and other local agencies did not employ persons who could or would ena'Vt the 
"generalist" helping role. 
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Policy Implications of Preliminary Process Findings 



A number of conclusions that are directly relevant for federal dissem- 
ination policies may be drawn from the above findings. First,, it is clear 
that the process is important in determining the organizational outcomes of 
a dissemination program. In particular, dissemination programs that do not 
encourage and support high levels of effort on the part of school staffs in 
engaging in a problem solving process are less likely to effect long-tenr 
organizational change. Since level of effort in the R&D Utilization pro- 
gram was significantly affected by the availability of release time paid 
for by the projects , it is clear that if a dissemination program seeks real 
educational impact, mechanisms for underwriting the necessary labor must be 
found. 

Second, the degree to which the process was of high quality (e.g., 
approximated theoretical ideals of rationality and/or participatory decision 
making) is of importance in determining the degree to which real change is 
effected, if the goal of a dissemination program is simply to implement a 
new activity or curriculum product in the short run, we need not perhaps pay 
a great deal of attention to the development of local capacity. If, however, 
our goal is to improve the functioning of schools and the long-range quality 
of their curriculum, we cannot afford to ignore school needs for assistance 
and training in problem solving. 

Finally, the role of the external linking agent who spends a consider- 
able degree of time with the school over a longer period of time, and who 
takes an active and instrumental change agent posture, is a significant 
feature of the process. It is important to emphasize that it is not simply 
the presence or availability of a generalist who provides only an extra pair 
of hands to support locally determined activities (although the significance 
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of such support should not in any way be underestimated). It is, in addition, 
the role of the linking agent as an initiator and as a facilitator of the 
process that is a key to both improved functioning of the internal team and 
increases in the scope of implementation. This finding further extends the 
potential policy dilemma that is inherent in all current dissemination pro- 
grams that employ linking agents. That dilemma is that linking agents appear 
to be effective only insofar as they provide their clients with a relatively 
substantial and continuing assistance. Nevertheless, in any federally funded 
effort that is not a demonstration program, the pressure to provide services 
to all comers is clearly present. Many observers have criticized the RDU 
program as : : an unrealistic model because of the external resources provided to 
the schools largely through intensive linking agent roles which inherently re- 
duce the potential scope of the client base. However, policy makers cannot 
and should not ignore the equity dilemmas that are eJmbedded in school improve- 
ment programs. The RDU program showo some evidence of a strategy that can ef- 
fectively stimulate change in schools that have multiple educational problems 
and limited internal resources for meeting those problems. If high levels of 
resources are needed to meet the needs of these .schools, the goals of educa- 
tional equity may indeed compete with the goal of equal access to resources. 

Summary and Caveats 

The results of our preliminary analysis suggest that the interventions 
that were part of the original RDU project design were largely effective. 
The underlying assumption that both the availability of high quality products, 
the development of an improved problem solving process inside the school, and 
the access to external technical assistance will promote school improvement is 
supported. In addition, the variables related to the RDU intervention are more 
powerful predictors of the success of the school improvement efforts in the 
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sites served by RDU than is the school's initial "readiness," as measured by 
previous experience with innovative efforts, with federally funded programs, 
with using externally developed R&D products, etc. (See Table 2.) 

It should be stressed, however, that these results are quite preliminary 
and have not been tested in our larger data base, which includes survey data 
from principals and teachers on a larger sample of RDU sites. A more sig- 
nificant conceptual limitation of the analysis presented here must be men- 
tioned. The above discussion emphasizes a view of school change that is pri- 
marily "technological" in nature (House, 1980). RDU strategies are viewed as 
"inputs," and the success of the school change program is viewed as the "out- 
put." We wish to emphasize that our qualitative data suggest that this view 
is highly simplistic, regardless of the predictive power that we have achieved 
with program variables. In the real world of the school, we have observed that 
much of the responsibility for the success of a change project is dependent not 
on factors that are predictable, or known at the beginning of the process, but 
which might be labeled "normal critical events." These key events occur with 
sufficient frequency in the cycle of school activities that they cannot (or 
should not) simply be thought of as sources of error in our regression equa- 
tions. * Among the most important and frequently occurring events were: turn- 
over of key administrators, both in the central office and in the school; 
strikes or prolonged contract negotiations; and reductions in f.orce. It should 
be emphasized that critical events do not necessarily impede a school change 
effort — even strikes may have positive impacts by increasing the cohesiveness 
of the school. Whether effects of events are good or bad, however, their im- 
pact upon the problem solving process and its outcomes are as strong as the 
findings presented here. Thus, in our later analyses we will move beyond a 
simple "technological" perspective on the problem of school improvement, 
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Table 3 

SIGNIFICANT PREDICTORS OF 
OUTCOMES OF RDU PROGRAMS* 

Readiness 

Centrality of problem (problem is viewed as the most important in the 
school by most staff) 

(2) Problem identification prior to involvement in program 
Li;iking Agent Behaviors : 

(1) Time : Amount of time spent with client school 

(2) Influence : Amount of influence over client activities and decision 

making 

(3) Process Role : Degree to which agent acted as a strong process facili- 

tator v 

Process Variables 

(1) Strong internal team 

(2) High teacher influence 

Product Variables 

(1) Empirical evidence of effective areas 

(2) Affects pupils directly 

(3) Product is complex (has many parts) 



♦Significant defined as entering at least threa of five regressions, 
with an F statistic significant at .10 or better, 
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and begin to analyze how other political, structural and cultural charac- 
teristics of the school affect the outcomes of the change process- 
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APPENDIX A 
A Note on Methods 

The analysis presented in this paper is based on data collected from 
schools using qualitative research methods. Approximately half of the 
schools were "case study sites, n and were visited regularly by a case study 
writer over the period of several years. The case study writers were em- 
ployed by the seven operational RDU projects rather than Abt Associates, 
but the AAI research team had some influence over the content and analysis 
of the case study data. The case study sites were not chosen by any statis- 
tical sampling scheme, but it is known that all projects selected the sites 
to be included in case studies before there was any data that would have al- 
lowed a prediction of "success. M 

The remainder of the sites were visited by AAI research team staff mem- 
bers. The number of person days at a site varied from four to seven. During 
the site visits, a topical guide was used as a basis for conducting open-ended 
interviews with school and district administrators, and a sample of between 
five and ten teachers and specialists at each site. Site visit notes were 
developed on the basis of the interview, and were written according to the 
topical inteview guide which reflected the variables included in this study. 
Sites were selected as follows: each project nominated their "worst" site, 
and the site that best exemplified what they were trying to do. Four more 
were sampled randomly from among the remaining non-case study sites in each 
project. 

In order to include the rich, qualitative data in our quantitative 
data base, each site was later subjected to a coding procedure, which in- 
volved having the individual most familiar with the site read through all 
materials available for that site (which included some early survey data, 




reports written by RDU project personnel, site reports and/or case studies, 
and any other materials or documents that were available) and answering over 
120 likert scale or dichotorrous scale questions. An inter-rater reliabil- 
ity check produced a score of .72 when disagreements based on "can't judge 
because of too little information" were eliminated (this elimination was done 
because one of the raters was more familiar with the site than the other). 

Later analysis in this project will be based not only on this data, but 
upon a survey of teachers and principals. These surveys focus more heavily 
upon the program client attitudes toward the services that they received in 
the program. In addition, the case studies and site visit materials will be 
analyzed using more traditional "wholistic" qualitative methods. 
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